Search for: All records

Recent advances in LLMs offer new opportunities for supporting student writing, particularly through real-time, composition-level feedback. However, for such support to be effective, LLMs need to generate text completions that align with the writer’s internal representation of their developing message, a representation that is often implicit and difficult to observe. This paper investigates the use of eyetracking data, specifically lookback fixations during pauses in text production, as a cue to this internal representation. Using eye movement data from students composing texts, we compare human-generated completions with LLM-generated completions based on prompts that either include or exclude words and sentences fixated during pauses. We find that incorporating lookback fixations enhances human-LLM alignment in generating text completions. These results provide empirical support for generating fixation-aware LLM feedback and lay the foundation for future educational tools that deliver real-time, composition-level feedback grounded in writers’ attention and cognitive processes. 

Classical serial models view the process of producing a text as a chain of discrete pauses during which the next span of text is planned, and bursts of activity during which this text is output onto the page or computer screen. In contrast, parallel models assume that by default planning of the next text unit is performed in parallel with previous execution. We instantiated these two views as Bayesian mixed-effects models across six sets of keystroke data from child and adult writers composing different types of multi-sentence text. We modelled interkey intervals with a single distribution, hypothesised by the serial processing account, and with a two-distribution mixture model that is hypothesised by the parallel-processing account. We analysed intervals occuring before-sentence, before word, and within word. Model comparisons demonstrated strong evidence in favour of the parallel view across all datasets. When pausing occurred, sentence initial inter-keystroke intervals were longer than word initial pauses. This is consistent with the idea that edges of larger linguistic units are associated with higher level planning. However, we found – across populations – that interkey intervals at word and even at sentence boundaries were often too brief to plausibly represent time to plan what was written next. Our results cannot be explained by the serial processing but are in line with the parallel view of multi-sentence text composition.